Image processing apparatus

ABSTRACT

An image processing apparatus includes a designating section for designating an arbitrary region or an arbitrary position of an image; a specifying section for specifying an object region which is present in the designated region or position, and which can additionally be in a vicinity of the designated region or position, from pixel information in the designated region or position; a determining section for determining an image region to be cut out from the image, based on the specified object region; and a cutting section for cutting out the determined image region from the image.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to an image processing apparatusfor use in computers for conducting image processing, word processors,portable information tools, copying machines, scanners, facsimiles orthe like. More specifically, the present invention relates to an imageprocessing apparatus enabling a user to designate the coordinates of anypoint on the image by a coordinate input apparatus such as a mouse, apen or a tablet, or an image processing apparatus capable ofphotoelectrically converting a printed image on a piece of paper or thelike with coordinates being designated in a different type of ink so asto input the image and the coordinates, wherein the image processingapparatus being capable of cutting out an object image with an arbitrarysize at an arbitrary position from the original image.

[0003] 2. Description of the Related Art

[0004] When an image including an object or a person's face of interestis cut out from the original image, the image is cut with a desired sizeusing a pair of scissors, a cutter or the like, in the case of aphotograph. In the case of an electronic image obtained by a CCD cameraor a scanner, however, the positions of two points are designated by acoordinate input device such as a mouse, using software for imageprocessing (e.g., the image processing software “PhotoShop” made byAdobe Inc.), and a rectangle having a diagonal between the two points isdesignated as a region.

[0005] In order to output a part of the original image, which includesan object of interest, as an image having a particular size, a portionhaving the object of interest at a well-balanced position is first cutout from the original image, and thereafter, is magnified/reduced to arequired size. In the case of a photograph, such magnification/reductionis conducted by, for example, a copying machine. In the case of anelectronic image, magnifying/reducing the image to a desired size can beeasily carried out. However, cutting out a portion having the object ofinterest at a well-balanced position must be conducted before suchmagnification/reduction.

[0006] Furthermore, in order to extract a region representing a person'sface except for hair (hereinafter, this portion is referred to as a“face skin”) from the original image, a face skin region which isvisually determined by an operator is painted out. In the case of anelectronic image, a pixel is designated by a coordinate input devicesuch as a mouse, and those pixels having a similar color to that of thedesignated pixel are combined to be extracted as one region (e.g.,“PhotoShop” as mentioned above). There is also a method as follows: thecolor distribution of a face skin is analyzed in advance to set aprobability density function. Then, the probability density of the inputpixels is obtained using values such as RGB (red, green, blue) valuesand HSV (hue, color saturation, brightness) values as arguments, therebydesignating those pixels having a probability equal to or higher than aprescribed value as a face-skin region (R. Funayama, N. Yokoya, H. Iwasaand H. Takemura, “Facial Component Extraction by Cooperative Active Netswith Global Constraints”, Proceedings of 13th International Conferenceon Pattern Recognition, Vol. 2, pp. 300-305, 1996).

[0007] Conventionally, in the case where a rectangle including aface-skin region in the image is determined, the rectangle is commonlydetermined visually by an operator.

[0008] Moreover, the central axis of a person's face has been commonlydetected based on the visual determination of an operator.

[0009] Another method for detecting the central axis of the face is asfollows: a skin-color portion of the face is extracted as a region, andthe region is projected to obtain a histogram. Then, the right and leftends of the face are determined from the histogram, whereby the linepassing through the center thereof is determined as the central axis ofthe face (Japanese Laid-Open Publication No. 7-181012).

[0010] Furthermore, respective vertical positions of the nose, the eyesand the mouth on the face have been commonly detected based on thevisual determination of an operator.

[0011] Another method is to match an image template of the nose with aninput image (*Face Recognition: Features versus Templates*, by R.Brunelli and T. Poggio, IEEE Transactions on Pattern Analysis andMachine Intelligence, Vol.15, No.10, pp.1042-1052, 1993). In thisarticle, a method for detecting the vertical positions by projecting agray-level image or an edge image to obtain a histogram, and examiningpeaks and valleys of the histogram, has also been proposed.

[0012] Moreover, the width of the face has been commonly detected basedon the visual determination of an operator.

[0013] Another method is as follows: a skin-color portion of the face isextracted as a region, and the region is projected to obtain ahistogram. Then, the right and left ends of the face are determined fromthe histogram, whereby the distance between the ends is obtained as thewidth of the face (Japanese Laid-Open Publication No. 7-181012).

[0014] As described above, in order to output a part of the originalimage, which includes a person's face of interest, as an image having aparticular size, a portion having the face at a well-balanced positionis first cut out from the original image, and thereafter, ismagnified/reduced to a required size. In the case of a photograph, suchmagnification/reduction is conducted by, for example, a copying machine.In the case of an electronic image, magnifying/reducing the image to adesired size can be carried out easily. However, cutting out a portionhaving the object of interest at a well-balanced position must beconducted before such magnification/reduction.

[0015] In the case of an electronic image, it is also possible for auser to adjust, in advance, the size of the face of the original imageto an appropriate size, move a frame on the screen according to thevisual determination of the user so that the face is located in thecenter, and output the image located within the frame. An apparatusachieving such an operation has been proposed in Japanese Laid-OpenPublication No. 64-82854.

[0016] In order to achieve improved visual recognition of a person'sface on a photograph or an image, the amount of exposure light forprinting is adjusted in the case of a photograph. For an electronicimage, there is software capable of conducting adjustment of contrast,tonality and brightness, edge sharpening, blurring processing and thelike (e.g., “PhotoShop” as mentioned above).

[0017] When an image including an object or a person's face of interestis cut out from the original image, the image is cut with a desired sizeusing a pair of scissors, a cutter or the like, in the case of aphotograph. However, using a pair of scissors, a cutter or the like tocut an image is actually time-consuming. Moreover, cutting a portionincluding the object or the face of interest at a well-balanced positionrequires much skill. When software for processing an electronic imageobtained by a CCD camera or converted by a scanner is utilized (e.g.,“PhotoShop” as mentioned above), the positions of two points are usuallydesignated by a coordinate input device such as a mouse, and a rectanglehaving a diagonal between the two points is designated as a region. Inthis case as well, cutting out a portion including an object or a faceof interest at a well-balanced position requires much skill.Furthermore, in the case where an object or a face of interest isoriginally located at the edge of the screen, and a portion includingthe object or the face at a well-balanced position in the center is tobe cut out from the image, it is necessary to first cut out the portionfrom the original image, and thereafter, move the position of the objector the face to the center of the resultant image.

[0018] As described above, in order to output a part of the originalimage, which includes an object of interest, as an image having aparticular size, a portion having the object of interest at awell-balanced position is first cut out from the original image, andthereafter, is magnified/reduced to a required size. In the case of aphotograph, such magnification/reduction is conducted by, for example, acopying machine. However, the image is not always cut to the same size.Therefore, in order to obtain an image with a desired size, atroublesome operation of calculating the magnification/reduction ratiois required. In the case of an electronic image, magnifying/reducing theimage to a desired size is easy. However, cutting out a portion havingthe object of interest at a well-balanced position must be conductedbefore such magnification/reduction. In short, at least two operationsare required to output an image having a particular size.

[0019] Furthermore, the above-mentioned method of painting out avisually determined face-skin region is troublesome regardless ofwhether an image to be processed is a photograph or an electronic image.Moreover, painting a portion at the boundary between the face skinregion and the other regions must be conducted extremely carefully. Inthe case of an electronic image, the above-mentioned method of combiningthose pixels having similar color to that of the designated pixel toextract them as one region (e.g. “PhotoShop” as mentioned above) hasbeen used. In this method, however, since the colors of the skin, thelip and the eyes are different, it is necessary to combine the resultsof several operations in order to extract the whole face-skin. Moreover,the skin color may be significantly uneven even in the same person dueto, for example, different skin shades or shadows. In this case as well,the results of several operations must be combined. Also described aboveis the method of designating those pixels having a probability equal toor higher than a prescribed value as a face-skin region (the above-citedreference by R. Funayama, N. Yokoya, H. Iwasa and H. Takemura).According to this method, however, a face-skin region might not besuccessfully extracted in the case where the image's brightness isextremely uneven due to the photographing conditions or the conditionsat the time of obtaining the image, or in the case where the color ofthe skin is different due to a racial difference.

[0020] As described above, when a rectangle including a face-skin regionis to be obtained, the rectangle has been commonly determined visuallyby an operator. However, such a method is troublesome regardless ofwhether an image to be processed is a photograph or an electronic image.

[0021] Moreover, in the above-mentioned method of detecting the centralaxis of a person's face from a histogram (Japanese Laid-Open PublicationNo. 7-181012), the correct central axis can only be detected in the casewhere the face is completely directed to the front, while the correctcentral axis can not be obtained in the case where the face is turnedeven slightly to either side.

[0022] Furthermore, according to the above-mentioned method of matchingan image template of the nose with an input image (the above-citedreference by R. Brunelli and T. Poggio), it is desirable that the sizeof the nose to be extracted is known. In the case where the size of thenose is not known, templates of various sizes must be matched with theinput image, requiring substantial time for calculation. Moreover,according to the above-mentioned method of detecting the verticalpositions by examining peaks and valleys of the histogram (theabove-cited reference by R. Brunelli and T. Poggio), the verticalpositions might not be correctly extracted, for example, in the casewhere the face skin region or the background is not known. In short,wrong extraction could occur without precondition.

[0023] Moreover, according to the above-mentioned method to detect awidth of the face (Japanese Laid-Open Publication No. 7-181012), a faceskin region should be correctly extracted based on the colorinformation. However, in the case where a background region includes acolor similar to that of the face skin, a region other than the faceskin region might be determined as a face skin, or a shaded portion inthe face skin region might not be determined as face skin. The detectedwidth of the face might be different depending upon whether or not theears can be seen on the image. Moreover, the detected width could belarger than the correct width in the case where the face is turnedtoward either side.

[0024] As described above, in order to output a part of the originalimage, which includes an object of interest, as an image having aparticular size, a portion having the object of interest at awell-balanced position is first cut out from the image, and thereafter,is magnified/reduced to a required size. In the case of a photograph,such magnification/reduction is conducted by, for example, a copyingmachine. However, the image is not always cut to the same size.Therefore, in order to obtain an image with a desired size, atroublesome operation of calculating the magnification/reduction ratiois required. In the case of an electronic image, magnifying/reducing theimage to a desired size can be easily carried out. However, cutting outa portion having the object of interest at a well-balanced position mustbe conducted before such magnification/reduction. In short, at least twooperations are required to output an image having a particular size.According to a somewhat automated method as described in JapaneseLaid-Open Publication No. 64-82854, the user adjusts, in advance, thesize of the face of the original image to an appropriate size, moves aframe on the screen according to the visual determination of the user sothat the face is located in the center, and output the image locatedwithin the frame. Alternatively, the user adjusts, in advance, the sizeof the face of the original image to an appropriate size, moves aT-shaped indicator on the screen according to the visual determinationof the user so that the ends of the horizontal line of the T-shapedindicator overlap the eyes, respectively, and then, outputs an imagewithin a rectangle defined with an appropriate margin from the T-shapedindicator.

[0025] The above-described operation of adjusting the amount of exposurelight for printing in order to achieve improved visual recognition of aperson's face on a photograph or an image requires much skill. For anelectronic image, there is software capable of conducting adjustment ofcontrast, tonality and brightness, edge sharpening, blurring processingand the like (e.g., “PhotoShop” as mentioned above), as described above.In this case as well, using such software requires much skill, andusually, various operations must be tried until a desired image isobtained.

SUMMARY OF THE INVENTION

[0026] According to one aspect of the present invention, an imageprocessing apparatus includes a designating section for designating anarbitrary region or an arbitrary position of an image; a specifyingsection for specifying an object region which is present in thedesignated region or position, and which can additionally be in avicinity of the designated region or position, from pixel information inthe designated region or position; a determining section for determiningan image region to be cut out from the image, based on the specifiedobject region; and a cutting section for cutting out the determinedimage region from the image.

[0027] In one example, the determining section includes a section foradjusting a size of the image region to a prescribed size.

[0028] In one example, the determining section includes a correctingsection for entirely correcting the designated image region orcorrecting only a part of the designated image region.

[0029] According to another aspect of the present invention, an imageprocessing apparatus includes a designating section for designating anarbitrary region or an arbitrary position of an image; an analyzingsection for analyzing a color distribution in the designated region orposition and in a vicinity of the designated region or position; aadjusting section for adjusting a condition for specifying a face imagewhich is present in the image, according to a result of the analysis; aspecifying section for specifying a face image region which is presentin the designated region or position, and which can additionally be inthe vicinity of the designated region or position, based on the adjustedcondition; a determining section for determining an image region to becut out from the image, based on the specified face image region; and acutting section for cutting out the determined image region from theimage.

[0030] In one example, the determining section includes a section foradjusting a size of the image region, using the region or the positiondesignated by the designating section as a reference.

[0031] In one example, the specifying section includes a section forapplying noise elimination or labelling to the specified face imageregion to produce a face mask; a section for vertically scanning theproduced face mask to obtain a sum of vertical differential luminancevalues of pixels in the image corresponding to the face mask to producea histogram; and a section for detecting a central axis of a face from aprofile of the produced histogram.

[0032] In one example, the specifying section includes a section forapplying noise elimination or labelling to the specified face imageregion to produce a face mask; a section for vertically scanning theproduced face mask to obtain a mean luminance value of pixels in theimage corresponding to the face mask to produce a histogram; and asection for detecting a vertical nose position from a profile of theproduced histogram.

[0033] In one example, the specifying section includes a section forapplying noise elimination or labelling to the specified face imageregion to produce a face mask; a section for horizontally scanning theproduced face mask to obtain a mean luminance value of pixels in theimage corresponding to the face mask to produce a histogram; and asection for detecting a vertical eye position from a profile of theproduced histogram.

[0034] In one example, the specifying section includes a section forapplying noise elimination or labelling to the specified face imageregion to produce a face mask; a section for horizontally scanning theproduced face mask to obtain a mean luminance value of pixels in theimage corresponding to the face mask to produce a histogram; and asection for detecting a vertical mouth position from a profile of theproduced histogram.

[0035] In one example, the specifying section further includes a sectionfor detecting a vertical eye position from the profile of the producedhistogram; and a section for obtaining a middle position of a regionbetween the detected vertical eye position and the detected verticalmouth position to detect a width of the face mask at the middleposition.

[0036] In one example, the determining section includes a section foradjusting a position of the image region, based on the face imageregion, a central axis of a face in the face image, a vertical noseposition of the face in the face image, a vertical eye position of theface in the face image, a vertical mouth position of the face in theface image, and a width of a face mask of the face image.

[0037] In one example, the determining section includes a section foradjusting a size of the image region, based on the face image region, acentral axis of a face in the face image, a vertical nose position ofthe face in the face image, a vertical eye position of the face in theface image, a vertical mouth position of the face in the face image, anda width of a face mask of the face image.

[0038] In one example, the determining section includes a correctingsection for entirely correcting the designated image region orcorrecting only a part of the designated image region.

[0039] Thus, the invention described herein makes possible the advantageof providing an image processing apparatus capable of photoelectricallyconverting a printed image on a piece of paper or the like withcoordinates being designated in a different type of ink so as to inputthe image and the coordinates, wherein the image processing apparatusbeing capable of cutting out an object image with an arbitrary size atan arbitrary position from the original image.

[0040] This and other advantages of the present invention will becomeapparent to those skilled in the art upon reading and understanding thefollowing detailed description with reference to the accompanyingfigures.

BRIEF DESCRIPTION OF THE DRAWINGS

[0041]FIG. 1 is a block diagram showing an image processing apparatusaccording to one example of the present invention;

[0042]FIG. 2 is a block diagram showing an image/coordinate inputapparatus in the image processing apparatus shown in FIG. 1;

[0043]FIG. 3 is a block diagram showing another image/coordinate inputapparatus in the image processing apparatus shown in FIG. 1;

[0044]FIG. 4 shows examples of a region of an object or a face in theimage designated by the user;

[0045]FIG. 5 shows examples of a position of the object or a face in theimage designated by the user;

[0046]FIGS. 6A through 6D show images illustrating the steps from theuser's designation to the extraction of an image;

[0047]FIG. 7 is a flow chart illustrating Image processing procedure 1conducted by the image processing apparatus of the example shown in FIG.1;

[0048]FIG. 8 is a diagram showing the pixels of an object region;

[0049]FIG. 9 is a diagram illustrating how a part of an image isattached to a document;

[0050]FIG. 10 is a diagram illustrating an example of extracting only aface-skin portion from the image including a person's face;

[0051]FIGS. 11A, 11B and 11C show the frequency histograms plotted withrespect to the hue, color saturation and brightness, respectively;

[0052]FIG. 12 is a flow chart illustrating Image processing procedure 3for producing an image representing a face skin region;

[0053]FIGS. 13A and 13B show input patterns designated by the user;

[0054]FIG. 14A shows an example of the image;

[0055]FIG. 14B is a graph showing the relationship between brightnessand frequency of the image of FIG. 14A;

[0056]FIG. 15 is a diagram illustrating an example of extracting only aface skin portion from the image including a person's face;

[0057]FIG. 16 is a diagram illustrating how the size of a window regionis gradually increased;

[0058]FIG. 17 shows an input pattern designated by the user;

[0059]FIG. 18 is a flow chart illustrating Image processing procedure 5for producing a face mask by the image processing apparatus of theexample shown in FIG. 1;

[0060]FIG. 19 illustrates how the face mask is produced;

[0061]FIG. 20 is a flow chart illustrating the process for detecting thecentral axis of the face;

[0062]FIG. 21 is a diagram illustrating the process for detecting thecentral axis of the face;

[0063]FIG. 22 is a flow chart illustrating Image processing procedure 6for detecting a vertical position of the nose by the image processingapparatus of the example shown in FIG. 1;

[0064]FIG. 23 is a diagram illustrating the process for detecting thevertical position of the nose;

[0065]FIG. 24 is a flow chart illustrating Image processing procedure 7for detecting a vertical position of the eyes by the image processingapparatus of the example shown in FIG. 1;

[0066]FIG. 25 is a diagram illustrating the process for detecting thevertical position of the eyes;

[0067]FIG. 26 is a flow chart illustrating Image processing procedure 8for detecting a vertical position of the mouth by the image processingapparatus of the example shown in FIG. 1;

[0068]FIG. 27 is a diagram illustrating the process for detecting thevertical position of the mouth;

[0069]FIG. 28 is a flow chart illustrating Image processing procedure 9for detecting a width of a face mask by the image processing apparatusof the example shown in FIG. 1;

[0070]FIG. 29 is a diagram illustrating the process for detecting thewidth of the face mask;

[0071]FIG. 30 is a flow chart illustrating Image processing procedure 10for cutting out a rectangular image from the original image by the imageprocessing apparatus of the example shown in FIG. 1;

[0072]FIG. 31 shows a sheet of an address book with a face image beingattached thereto; and

[0073]FIG. 32 is a diagram illustrating the process for correcting animage.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0074] Hereinafter, the present invention will be described by way ofillustrative examples with reference to the accompanying drawings. Thesame reference numerals designate the same component.

[0075]FIG. 1 is a block diagram showing an image processing apparatusaccording to one example of the present invention. An image to beprocessed and coordinates required for the processing are input by animage/coordinate input apparatus 1-1. In the case where the image is ina digital form, the image is directly stored in an input image storingsection 1-2-1 of a storage apparatus 1-2. In the case where the inputimage is an analog form, the image is converted into a digital form, andthe resultant image is stored in the input image storing section 1-2-1.The input coordinates are stored in an input coordinate storing section1-2-2. An image processing section 1-3 uses the stored image andcoordinates as input information to conduct an appropriate imageprocessing in an operation region of a memory within the imageprocessing section 1-3. Thereafter, the image processing section 1-3stores the resultant image and coordinates in an output image storingsection 1-2-3 and an output coordinate storing section 1-2-4 of thestorage apparatus 1-2, respectively. After undergoing processing, theresultant image can be sent to an image output apparatus 1-4, whereby acopy of the resultant image can be made.

[0076]FIGS. 2 and 3 are diagrams illustrating in detail theimage/coordinate input apparatus 1-1 shown in FIG. 1.

[0077] The image/coordinate input apparatus 1-1 in FIG. 1 separatelyincludes an image input apparatus 2-1 and a coordinate input apparatus2-2, as shown in FIG. 2. The input image from the image input apparatus2-1 is stored in the input image storing section 1-2-1 of the storageapparatus 1-2, whereas the input coordinates from the coordinate inputapparatus 2-2 are stored in the input coordinate storing section 1-2-2of the storage apparatus 1-2. For example, a camera capable of directlyinputting a digitized image by a solid-state image sensing device (CCD;charge coupled device); an apparatus capable of digitizing a photographor a scanner which can input printed matters; or an apparatus forholding a digitized image such as equipment connected to a network, likethe internet, and a magnetic storage apparatus may be used as the imageinput apparatus 2-1. As the coordinate input apparatus 2-2, a mousecapable of inputting coordinates with a pointer displayed on a display,a track ball, a pen-type apparatus, a pen-type coordinate inputapparatus using a tablet, a coordinate input apparatus using a finger,or the like may be used.

[0078] The image/coordinate input apparatus 1-1 in FIG. 1 includes animage reading apparatus 2-3 and an image/coordinate separation apparatus2-4, as shown in FIG. 3. This type of the image/coordinate inputapparatus 1-1 is used in the case where both an image including anobject to be processed and input coordinates are present on a singleimage. For example, in the case where a line or a point representing thecoordinates is drawn in a particular color on a photograph, only acomponent of that color is extracted to obtain a separate image.Thereafter, the position of the point or the line is analyzed from theseparate image, whereby the coordinates are extracted.

[0079]FIG. 4 shows examples of a region of the object in the imagedesignated by the user. First, an image and a pattern indicated by asolid line or points, as shown in FIG. 4, are input to theimage/coordinate input apparatus 1-1 (FIG. 1). In the case of arectangular pattern 4-1, the coordinates of two points, that is, thecoordinates of the upper left point and the lower right point of thepattern are used as the input coordinates. In the case of a pattern 4-4,4-10, 4-11, 4-12, 4-13 or 4-14, the coordinates of the upper left pointand the lower right point of a rectangle circumscribing the inputpattern (i.e., such a rectangle as shown by a dotted line on each image)are used as the input coordinates. In the case of the other patterns,two coordinates defining a rectangle circumscribing the input patterncan be used as the input coordinates. However, in the case of a line ordot pattern, that is, in the case of a pattern 4-2, 4-3, 4-5 or 4-6, norectangle circumscribing the pattern could be obtained. Otherwise, sucha rectangle that has an extremely large aspect ratio would be obtained.In such a case, an appropriate rectangle will be set according to a meanaspect ratio of the object (this rectangle will be set as a square whenthe object is not known). In the case of a pattern 4-2, for example, theobject is a person's face and a rectangle circumscribing the inputpattern is extremely long in the longitudinal direction (or the inputpattern is a vertical straight line and no rectangle circumscribing theinput pattern can be obtained). In such a case, a rectangle as shown bya dotted line is set. In other words, a rectangle horizontallymagnified/reduced from the rectangle circumscribing the input pattern isobtained by multiplying the length of the rectangle circumscribing theinput pattern by a prescribed ratio. Furthermore, the coordinates of theupper left point and the lower right point are used as the inputcoordinates. In the case of a pattern 4-7, 4-8 or 4-9, a rectanglelongitudinally and laterally magnified from the rectangle circumscribingthe input pattern by respective prescribed ratios is set, and thecoordinates of two points of the rectangle are used as the inputcoordinates.

[0080]FIG. 5 shows examples of a position of the object designated bythe user. In the case where the user designates a point such as apattern 5-1, the coordinates of that point can be used as the inputcoordinates. In the case where the user designates a pattern other thanthe point such as a pattern 5-2, the center of a circumscribed rectanglecan be used as the input coordinates.

[0081] Image Processing Procedure 1

[0082] Image processing procedure 1 conducted by the image processingapparatus of the present example will now be described with reference tothe flow chart of FIG. 7.

[0083] First, using the image/coordinate input apparatus 1-1 (FIG. 1),the user roughly designates a region of the object in the image storedin the input image storing section 1-2-1, as shown in FIG. 4, or roughlydesignates a position of the object, as shown in FIG. 5. FIGS. 6Athrough 6D show images illustrating the steps from the user'sdesignation to the extraction of an image. When a region 6-1-1 isdesignated by the user (Step S1-1), as shown in FIG. 6A, the imageprocessing section 1-3 obtains a rectangular region 6-1-2 reduced fromthe rectangle circumscribing the input pattern by an appropriate ratio,and stores the region 6-1-2 as a set region in the input coordinatestoring section 1-2-2 (Step S1-7). As shown in FIG. 6B, when a position6-2-1 is designated by the user (Step S1-2), the image processingsection 1-3 obtains an appropriate rectangular region 6-2-2 centeredaround the designated position 6-2-1 (Step S1-3), and stores the region6-2-2 in the input coordinate storing section 1-2-2 (Step S1-7).

[0084] The image processing section 1-3 (FIG. 1) utilizes the operationregion of the memory within the image processing section 1-3 to storethe color information of the pixels included in the rectangular region6-1-2 or 6-2-2 (Step S1-4), and sets the rectangular region 6-1-2 or6-2-2 as an initial value of the object region (Step S1-5).

[0085]FIG. 8 shows the pixels in the object region. The image processingsection 1-3 finds a pixel 8-2 adjacent to the object region 8-1. Whenthe pixel 8-2 satisfies at least one of the following two conditions(Step S1-6), the pixel 8-2 is added to the object region (Step S1-9):

[0086] 1. the color difference between the pixel of interest and anadjacent pixel in the object region is within a prescribed range; and/or

[0087] 2. the color difference between the pixel of interest and a pixelstored in Step S1-4 is within a prescribed range.

[0088] The image processing section 1-3 examines all of the pixelsadjacent to the object region in terms of the above two conditions. Thisoperation is repeated until no pixel can be added to the object region.Then, as shown in FIG. 6C, the image processing section 1-3 obtains afinal object region 6-3-1 (Step S1-8). It should be noted that, althoughvarious indices of the color difference have been proposed, a Godlove'scolor-difference formula as shown in “Improved Color-Difference Formulawith Applications to the Perceptibility and Acceptability of Fadings”,I. H. Godlove, J. Opt. Soc. Am., 41, 11, pp. 760-772, 1951 may be used.

[0089] The image processing section 1-3 expresses the area of the objectregion 6-3-1 as the number of pixels included in the object region6-3-1. Then, as shown in FIG. 6D, the image processing section 1-3obtains a rectangular region 6-3-3 centered around the center of gravityof the object region 6-3-1 and having an area corresponding to aprescribed percentage (e.g., 30%) of the total area of the rectangularregion 6-3-3. Thereafter, the image processing section 1-3 cuts out therectangular region 6-3-3 from the original image. The shape of therectangular region 6-3-3 may be square. Alternatively, the shape of therectangular region 6-3-3 may be set as appropriate depending uponapplications. For example, the rectangular region 6-3-3 may be set tohave a ratio of 4:3 according to the aspect ratio of a televisionscreen, or may be set to have a ratio of 16:9 according to the aspectratio of a high-definition television screen. It should be noted that,although the rectangular region is centered around the center of gravityof the object region in the above description, the position of thecenter of gravity in the rectangular region may be shiftedlongitudinally and laterally depending upon the application.

[0090] A method for obtaining the center of gravity is described in, forexample, “Robot Vision” by M. Yachida, Shohkohdo, ISBN4-7856-3074-4C3355, 1990. A part of the image can be cut out from the original image,based on the coordinates of the rectangular region.

[0091] Image Processing Procedure 2

[0092] The image processing section 1-3 magnifies or reduces the imagewhich has been cut out according to Image processing procedure 1, to anappropriate size, and stores the resultant image in the output imagestoring section 1-2-3 of the storage apparatus 1-2. The image processingsection 1-3 may utilize the stored image for any appropriateapplications. For example, an image 9-1 including an automobile andobtained by a digital camera, as shown in FIG. 9, is stored in the inputimage storing section 1-2-1. Then, a part of the image including onlythe automobile is cut out from the input image. Thereafter, this part ofthe image is attached to a report 9-2 having a prescribed format and aframe for a prescribed image size. The resultant report 9-2 is stored inthe output image storing section 1-2-3.

[0093] Image Processing Procedure 3

[0094] Before Image processing procedure 3, the color distribution of aperson's face skin is analyzed in advance according to the followingprocedures:

[0095] 1. the face-skin portion is manually extracted from a face image10-1 to produce a face-skin image 10-2 (FIG. 10);

[0096] 2. a face-skin image is similarly produced for a plurality ofdifferent persons;

[0097] 3. frequency histograms are plotted with respect to the hue (FIG.11A, 11-1-1), color saturation (FIG. 11B, 11-2-1) and brightness (FIG.11C, 11-3-1) of the pixels of each of the face-skin images to obtain thecolor distribution; and

[0098] 4. for each histogram, the mean and variance of the distributionare obtained, and such a normal probability density function (11-1-2,11-2-2, 11-3-2) that best fits the distribution is obtained.

[0099] Thus, the color distribution of the face skin can be expressed bythe normal probability density functions (P_(hue)(hue), P_(sat)(sat) andP_(val)(val)) of the hue, color saturation and brightness, each functionhaving two arguments: the mean and variance (μ_(hue), σ² _(hue);μ_(sat), σ² _(sat); and μ_(val), σ² _(val), respectively). In thisspecification, each of the normal probability density functions isreferred to as a skin-region probability density function. Eachskin-region probability density function is expressed by the followingexpressions:

P _(hue)(hue)˜N(μ_(hue), σ² _(hue))  (1)

P _(sat)(sat)˜N(μ_(sat), σ² _(sat))  (2)

P _(val)(val)˜N(μ_(val), σ² _(val))  (3)

[0100] When the calculated mean and variance are applied to the normaldistribution, those values which are significantly different from a meanvalue, if any, would result in a greater estimation of the variance thanthe actual variance. Even a few values would cause such an estimation.For example, in the case of the hue distribution histogram as shown inFIG. 11A, most of the pixels are distributed within about ±30 of about20. In this histogram, values such as 100 and −150 would result in agrater estimation of the variance. Therefore, in order to obtain anormal distribution curve (a probability density function) which can beapplied to a more accurate distribution, it would be better to firstremove those pixels having such values, and thereafter, calculate themean and variance.

[0101] The image processing section 1-3 stores each of the normalprobability density functions in advance, and processes the image storedin the input image storing section 1-2-1 according to the flow chart ofFIG. 12. In Step S1-0, the image processing section 1-3 sets an originalprocessing region, based on the user input. In the case where a pattern(region) 9-1 as shown in FIG. 13A is input from the image/coordinateinput apparatus 1-1 to the input coordinate storing section 1-2-2, theimage processing section 1-3 sets a processing region 9-2 of the imagestored in the input image storing section 1-2-1 in such a way asdescribed above. In the case where a pattern (position) 9-4 as shown inFIG. 13B is input, the image processing section 1-3 sets a processingregion 9-5 (Step S1-0). The image processing section 1-3 substitutes ahue value, a color-saturation value and a brightness value of each pixelin the respective normal probability density functions obtained asdescribed above, so as to obtain the respective probabilities. Such apixel that has a value equal to or higher than a prescribed probabilitywith respect to each of the hue, color saturation and brightness isdetermined as an original probable face-skin pixel (Step S2-1). At thistime, the prescribed probability should be set to a small value such as5% so that as many pixels as possible may be selected as a probableface-skin pixel. Thus, any pixels which possibly correspond to theface-skin portion are determined as original probable face-skin pixels.Thereafter, the image processing section 1-3 calculates the mean andvariance of each of the hue, color saturation and brightness (StepS2-2). In the foregoing description, an original probable face-skinpixel is selected based on the probabilities of the hue, colorsaturation and brightness. However, it may also be effective to adjusteach threshold to a value close to the pixel value of theabove-mentioned prescribed probability, depending upon thecharacteristics of an imaging system.

[0102] Provided that the mean and variance of the hue, colordistribution and brightness thus calculated are μ′_(hue), σ^(2′) _(hue);μ′_(sat), σ^(2′) _(sat); and μ′_(val), σ^(2′) _(val), respectively,corresponding probability density functions P′_(hue)(hue), P′_(sat)(sat)and P′_(val)(val) having these arguments can be expressed by thefollowing expressions:

P′ _(hue)(hue)˜N(μ′_(hue), σ^(2′) _(hue))  (4)

P′ _(sat)(sat)˜N(μ′_(sat), σ^(2′) _(sat))  (5)

[0103]  P′ _(val)(val)˜N(μ′_(val), σ^(2′) _(val))  (6)

[0104]

[0105] Using these probability density function, the image processingsection 1-3 selects face-skin pixels according to the followingprocedures:

[0106] 1. first, all of the pixels in the image are set as initialvalues, and any pixels having a value equal to or lower than aprescribed probability (P′_(hue)(hue)) calculated from a hue value as anargument are removed (Step S2-3);

[0107] 2. next, any pixels having a value equal to or lower than aprescribed probability (P′_(sat)(sat)) calculated from acolor-saturation value as an argument are removed (Step S2-4); and

[0108] 3. finally, any pixels having a value equal to or lower than aprescribed probability (P′_(val)(val)) calculated from a brightnessvalue as an argument are removed (Step S2-5).

[0109] As a result, a face-skin region is specified (Step S2-6).

[0110] The lower limit of each probability is set higher than they wereset when the original probable face-skin pixels were obtained. Forexample, provided that the previous threshold of the probability is 5%as described above, the threshold may be set to 30%. As a result, moreaccurate extraction can be carried out. More specifically, any pixelsthat have been wrongly extracted as not being noise based on the 5%threshold, would be removed based on the 30% threshold.

[0111] In the foregoing description, selection of the pixelscorresponding to the face-skin portion is conducted based on theprobabilities. However, it may also be effective to adjust eachthreshold to a value close to the pixel value of the above-mentionedprescribed probability, depending upon the characteristics of an imagingsystem. For example, as can be seen from FIG. 14A, the face skin and thehair of an image 14-1 have different brightnesses. FIG. 14B is ahistogram showing the brightness versus frequency of the image of FIG.14A. As shown in FIG. 14B, a peak 14-2 representing the hair appears ata lower value of the brightness, whereas a peak 14-3 representing theface-skin region appears at a relatively higher value of the brightness.Provided that a peak value is simply selected as a threshold of thebrightness probability of the image 14-1, the peak value 14-2 might beset as a threshold, whereby those pixels corresponding to a part of thehair might be selected as the pixels corresponding to the face skin. Insuch a case, such an algorithm as an Ohtsu's discriminant analysismethod (which is described in the above-cited reference: “Robot Vision”by M. Yachida) may be applied to a value equal to or lower than anappropriate brightness value to set a more appropriate value 14-5 as thebrightness threshold.

[0112] By updating the skin region probability density functions asappropriate in such a manner as described above, an image 12-3representing a face-skin region can be obtained from an image 12-1, asshown in FIG. 15 (Step S2-6). The image 12-3 thus obtained has a smalleramount of noise, as compared to an image 12-2 conventionally extractedusing a fixed function.

[0113] Image Processing Procedure 4

[0114] Image processing procedure 4 is conducted after the imagerepresenting the face skin-region is obtained according to Imageprocessing procedure 3. Referring to an image 16-1 in FIG. 16, in thecase where only a position 16-1-0 is designated by the user, the imageprocessing section 1-3 sets the smallest rectangle 16-1-1 centeredaround the designated point, and sets a region 16-1-3 located betweenthe rectangle 16-1-1 and a slightly larger rectangle 16-1-2 as aninitial window region. The image processing section 1-3 graduallymagnifies the window region 16-1-3 as shown in images 16-2 and 16-3,until one of the four sides of the outer rectangle 16-1-2 of the windowregion 16-1-3 reaches the edge of the input image. Thereafter, the imageprocessing section 1-3 calculates the dispersion of the pixels of thewindow region 16-1-3 in the image representing the face-skin region. Thelargest dispersion will be calculated when both the face skin and thecontour of a part other than the face skin appear in the window regionas shown in an image 16-4. Accordingly, during the operation ofgradually magnifying the window region 16-1-3, the image processingsection 1-3 determines the outer rectangle 16-1-2 corresponding to thelargest dispersion, as a rectangle including the face skin region.

[0115] As shown in FIG. 17, in the case where a region 15-1, not aposition, is designated by the user, the image processing section 1-3magnifies or reduces an outer rectangular defining a window region by anappropriate ratio to the size smaller than that of a rectangle 15-2obtained from the designated region 15-1. Thus, the smallest rectangle15-3 is obtained, whereby an initial window region is set such that theouter rectangle defining the window region corresponds to the rectangle15-3. Thereafter, the image processing section 1-3 gradually magnifiesthe window region, until an inner rectangle of the window region becomeslager than a rectangle 15-4 magnified by an appropriate ratio from therectangle 15-2. The image processing section 1-3 then calculates thedispersion of the pixels within the window region in a similar manner,and determines the outer rectangle corresponding to the largestdispersion, as a rectangle including the face-skin region. It should benoted that, provided that the region designated by the user is onlyslightly shifted from the face region, the rectangle magnified by anappropriate ratio from the rectangle obtained from the designated regionmay be determined as the rectangle including the face-skin region.

[0116] Image Processing Procedure 5

[0117]FIG. 18 is a flow chart showing Image processing procedure 5conducted by the image processing section 1-3. The image processingsection 1-3 processes an input color image 17-1 shown in FIG. 19according to Image processing procedure 4 to obtain a rectangleincluding a face-skin region 17-2. The image processing section 1-3processes that rectangle according to Image processing procedure 3 toobtain an image 17-3 representing a face skin region as shown in FIG.19. The image processing section 1-3 combines the pixels connected toeach other in the face-skin region image 17-3 to produce a label image.The image processing section 1-3 then extracts only a label regionhaving the largest area from the produced label image, and forms abinary image 17-4 from the label region (Step S3-1). Regarding the image17-4, the image processing section 1-3 replaces black pixels (holes)surrounded by white pixels with white pixels to fill the holes. As aresult, an image 17-5 is formed (Step S3-2). The image processingsection 1-3 first reduces the size of the image 17-5 once (Step S3-3),and again produces a label image. The image processing section 1-3extracts only a label region having the largest area from the labelimage (Step S3-4). After magnifying the resultant image n times (StepS3-5), the image processing section 1-3 reduces the size of the image ntimes (Step S3-6), and extracts only a label region having the largestarea from the resultant label image (Step S3-7). Thus, a face mask 17-6is obtained.

[0118] In the above steps, n should be set to, for example, 3 or 4depending upon the size, characteristics or the like of the image. Themagnifying and reducing processing as described above is described inthe above-cited reference: “Robot Vision” by M. Yachida.

[0119] The face mask 17-6 thus obtained is used to define the range tobe subjected to the processing according to the flow chart shown in FIG.20. The image processing section 1-3 extracts only luminance componentsfrom the input color image 17-1 to obtain a gray-level image 17-2 (StepS4-1). At the same time, the image processing section 1-3 produces theface mask 17-6 according to the flow chart in FIG. 18 (Step S3-0). Theimage processing section 1-3 differentiates the gray-level image 17-2 ina vertical direction with respect to the white pixels in the face mask17-6 to obtain a differentiated image 17-7 (Step S4-2). In the image17-7, those pixels corresponding to the black pixels in the face mask17-6 are set to zero. Such a differentiated image is commonly obtainedby using, for example, a Prewitt's operator (the above-cited reference:“Robot Vision” by M. Yachida).

[0120] The image processing section 1-3 projects the image 17-7 in avertical direction to obtain a histogram 17-8 (Step S4-3). A verticalaxis of the histogram 17-8 shows the sum of the pixel values of theimage 17-7 at a corresponding horizontal position. Referring to FIG. 21,the image processing section 1-3 sets such a vertical axis 21-1 a thathorizontally divides the histogram 21-1 into two regions: right and leftregions. The image processing section 1-3 obtains such an axis 21-2 thathas the smallest value of SSDS given by the following expression:${SSDS} = {\sum\limits_{i = 1}^{{({a - i})} > {a_{\min}\quad {and}\quad {({a + i})}} < a_{\max}}\left\{ \left( {{f\left( {a - i} \right)} - {f\left( {a + i} \right)}} \right)^{2} \right\}}$

[0121] where a indicates a position of the axis 21-1 a, a_(min)indicates a left end of the histogram, a_(max) indicates a right end ofthe histogram, and f(s) indicates a height of the histogram (Step S4-4).Then, the image processing section 1-3 sets the position 21-2 as acentral axis 21-3 of the face.

[0122] Image Processing Procedure 6

[0123]FIG. 22 is a flow chart illustrating Image processing procedure 6performed by the image processing section 1-3. The image processingsection 1-3 produces the gray-level image 17-2 and the face mask 17-6based on the image 17-1 as shown in FIG. 23 (Steps S4-1 and S3-0). Theimage processing section 1-3 horizontally scans only the gray-levelimage within the face mask 17-6 to produce a histogram 18-1 projecting amean luminance value (Step S5-1). The image processing section 1-3 thenproduces a histogram 18-2 having a reduced resolution from the histogram18-1 (Step S5-2), and searches for a peak position 18-2-1 approximatelyin the middle of the lower-resolution histogram 18-2 (Step S5-3). In thecase where no peak is found (Step S5-6, No), the image processingsection 1-3 sets the position in the middle of the histogram as avertical nose position (Step S5-5). In the case where any peak is found(Step S5-6, Yes), the image processing section 1-3 scans a region aroundthe position of the histogram 18-1 corresponding to the detected peak ofthe lower-resolution histogram 18-2, in order to search for a peakposition 18-3-1 (Step S5-4). The image processing section 1-3 sets thispeak position 18-3-1 as the vertical nose position (Step S5-0).

[0124] Image Processing Procedure 7

[0125]FIG. 24 is a flow chart illustrating Image processing procedure 7conducted by the image processing section 1-3. The image processingsection 1-3 produces a horizontal histogram 25-5 as shown in FIG. 25according to Image processing procedure 6 (Step S5-10). Using thishistogram 25-5, the image processing section 1-3 scans a region 25-1above a vertical nose position 25-6 detected in Image processingprocedure 6 to detect the deepest two valleys 25-2 and 25-3 (Step S6-1).In the case where the two valleys are both detected (Step S6-3), theimage processing section 1-3 sets the lower one of the valleys, that is,the valley 25-3 as a vertical position 25-7 of the eyes (Step S6-2). Inthe case where only one valley is detected (Step S6-4), the imageprocessing section 1-3 sets the detected valley as the vertical eyeposition (Step S6-5). In the case where no valley is detected, the imageprocessing section 1-3 sets the position in the middle of the regionbetween the vertical nose position and the upper end of the histogram25-5 as the vertical eye position (Step S6-6).

[0126] Image Processing Procedure 8

[0127]FIG. 26 is a flow chart illustrating Image processing procedure 8conducted by the image processing section 1-3. The image processingsection 1-3 produces a horizontal histogram 26-1 as shown in FIG. 27according to Image processing procedure 6 (Step S5-10). Using thehistogram 26-1, the image processing section 1-3 scans a region 26-3below the vertical nose position 26-2 detected in Image processingprocedure 6 to detect the deepest three valleys 26-4, 26-5 and 26-6(Step S7-1). In the case where the three valleys are detected (StepS7-2), the image processing section 1-3 sets the middle one of thevalleys, that is, the valley 26-5 as a vertical position 26-7 of themouth (Step S7-5), as shown in an image 26-8.

[0128] In the case where only two valleys are detected (Step S7-3), theimage processing section 1-3 first detects the widths of a face mask26-11 at the two valleys. Then, the image processing section 1-3calculates the ratio of the width 26-10 of the face mask 26-11 at thelower valley to the width 26-9 at the upper valley. In the case wherethe calculated ratio is higher than a prescribed value (e.g., 0.7) (StepS7-6), the image processing section 1-3 sets the position of the uppervalley as a vertical mouth position (Step S7-9). Otherwise, the imageprocessing section 1-3 sets the position of the lower valley as thevertical mouth position (Step S7-10).

[0129] In the case where only one valley is detected (Step S7-4), theimage processing section 1-3 sets the position of the detected valley asthe vertical mouth position (Step S7-7).

[0130] In the case where no valley is detected, the image processingsection 1-3 sets the position in the middle of the region between thevertical nose position and the lower end of the histogram 26-1 as thevertical mouth position (Step S7-8).

[0131] Image Processing Procedure 9

[0132]FIG. 28 is a flow chart illustrating Image processing procedure 9conducted by the image processing section 1-3. As shown in FIG. 29, aface mask 28-1, a vertical eye position 28-2 and a vertical mouthposition 28-3 are obtained according to Image processing procedures 7and 8 (Steps S3-0, S6-0 and S7-0). The image processing section 1-3horizontally scans the pixels from the vertical eye position 28-2 to thevertical mouth position 28-3 in order to obtain a width of the face mask28-1. The image processing section 1-3 obtains a width in the middle ofthe region between the vertical positions 28-2 and 28-3 as a width 28-4of the face (Step S29-1).

[0133] Image Processing Procedure 10

[0134]FIG. 30 is a flow chart illustrating Image processing procedure 10conducted by the image processing section 3-1. the face mask, thecentral axis of the face, the vertical eye position, the vertical mouthposition, and the width of the face are detected according to Imageprocessing procedures 5, 6, 7, 8 and 9. The distance between the eyesand the mouth can be obtained from the vertical eye position and thevertical mouth position. Using such information, the image processingsection 1-3 cuts out an image which includes a face having anappropriate size and located at a well-balanced position in thehorizontal and vertical directions, from the original image.

[0135] First, the image processing section 1-3 determines whether or notthe detected width of the face is reliable. The width of the face isdetected according to Image processing procedure 9, and the central axisof the face is detected according to Image processing procedure 5.Accordingly, the width of the face is divided into two widths by thecentral axis. A width on the left side of the central axis is hereinreferred to as a left-face width, whereas a width on the right side ofthe central axis is herein referred to as a right-face width. The imageprocessing section 1-3 verifies that the left-face width and theright-face width are not zero (Step S10-1). Then, the image processingsection 1-3 calculates the ratio of the left-face width to theright-face width to determine whether or not the calculated ratio iswithin a prescribed threshold-range (Step S10-2). In the case where theratio is not within the threshold-range (Step S10-2, Yes), the imageprocessing section 1-3 determines that the detected width of the face isnot reliable, and determines a rectangle to be cut out from the detectedeye-mouth distance (Step S10-6). More specifically, the image processingsection 1-3 sets the intersection of the central axis of the face andthe vertical nose position as a reference point. Then, the imageprocessing section 1-3 calculates a rectangle centered around thereference point and having a width and length each calculated as aproduct of the eye-mouth distance and a respective prescribed ratio(Step S10-6). Thus, the rectangle to be cut out is obtained.

[0136] In the case where the width of the face is reliable (Step S10-2,No), the image processing section 1-3 determines whether or not thedetected eye-mouth distance is reliable (Step S10-3). The imageprocessing section 1-3 calculates the ratio of the detected eye-mouthdistance to the length of the detected rectangle circumscribing apattern designated by the user, and determines whether or not thecalculated ratio is within a prescribed threshold-range (Step S10-3,No). Note that in the case where a position, not a region, is designatedby the user, the image processing section 1-3 calculates the ratio ofthe detected eye-mouth distance to a rectangle reduced by a prescribedratio from the face-skin region obtained according to Image processingprocedure 4. In the case where the ratio is not within thethreshold-range, the image processing section 1-3 determines that thedetected vertical eye position and the detected vertical mouth position(and the detected eye-mouth distance) are not reliable, and determines arectangle to be cut out from the detected width of the face. Morespecifically, the image processing section 1-3 sets as a reference pointthe intersection of the detected central axis of the face and thevertical center line of the rectangle circumscribing the patterndesignated by the user. Then, the image processing section 1-3calculates a rectangle centered around the reference point and having awidth and length each calculated as a product of the width of the faceand a respective prescribed ratio. Thus, the rectangle to be cut out isobtained (Step S10-5).

[0137] In the case where both the width of the face and the eye-mouthdistance are reliable (Step S10-3, Yes), the image processing section1-3 determines a rectangle to be cut out from these two values. Morespecifically, the image processing section 1-3 sets the intersection ofthe detected central axis of the face and the vertical nose position asa reference point, and calculates weighted arithmetic mean values byrespectively multiplying the width of the face and the eye-mouthdistance by a prescribed ratio. Then, the image processing section 1-3calculates a rectangle centered around the reference point and having awidth and length each calculated as a product of the respectivecalculated arithmetic mean value and a respective prescribed ratio (StepS10-4). Thus, a rectangle to be cut out is obtained.

[0138] Finally, the image processing section 1-3 calculates the ratio ofthe size of the rectangle thus obtained to the size of the rectanglecircumscribing the pattern designated by the user, and determineswhether or not the calculated ratio is within a prescribedthreshold-range (Step S10-7). In the case where the ratio is not withinthe threshold-range, the image processing section 1-3 determines thatthe obtained rectangle is not appropriate, and determines a rectanglefrom the pattern designated by the user. More specifically, in the casewhere a region is designated by the user, the image processing section1-3 sets the center of a rectangle circumscribing the region as areference point. Then, the image processing section 1-3 calculates arectangle centered around the reference point and having a width andlength each calculated as a product of the length of the circumscribingand a respective prescribed ratio (Step S10-8). Thus, the rectangle tobe cut off is obtained. In the case where a position is designated bythe user, the center of the rectangle including a face skin regionobtained according to Image processing procedure 4 is used as areference point, and similar processing is carried out to obtain arectangle to be cut out.

[0139] Image Processing Procedure 11

[0140] The image processing section 1-3 magnifies or reduces the faceimage which is cut out according to Image processing procedure 10 to anappropriate size, and stores the resultant image in the output imagestoring section 1-2-3 of the storage apparatus 1-2. The image processingsection 1-3 can utilize the stored face image for appropriateapplications such as an address book in a portable information tool. Forexample, the image processing section 1-3 stores an image of a personobtained by a digital camera, such as an image 30-1 as shown in FIG. 31,in the input image storing section 1-2-1, and roughly designates aportion in and around the face using the image coordinate inputapparatus 1-1. Then, the image processing section 1-3 cuts out an imageincluding the face at a well-balanced position from the original imageaccording to Image processing procedure 10, and magnifies or reduces theresultant image to fit a prescribed frame. Thus, the resultant image isattached to a document, as shown by an image 30-2 of FIG. 31. The image30-2 is a sheet of the address book with the face image being attachedthereto.

[0141] Image Processing Procedure 12

[0142] A face mask is obtained according to Image processing procedure12. In order to improve the visual recognition of the face in the image,the image processing section 1-3 of the present example appropriatelyprocesses only a portion of the input image corresponding to awhite-pixel region of the face mask to make the image characteristics ofthe face-skin region and the other regions different from each other.Alternatively, in order to improve the visual recognition of the face inthe image, the image processing section 1-3 may appropriately processonly a portion of the input image corresponding to a black-pixel regionof the face mask to make the image characteristics of the face regionand the other regions different from each other.

[0143] For example, FIG. 32 is a diagram illustrating the imagecorrection processing. In the case where a face mask 31-2 is obtainedfrom an input image 31-1, the image processing section 1-3 reduces thesharpness of the portion of the input image corresponding to theblack-pixel region of the face mask 31-2, using a Gaussian filter or anaveraging filter. As a result, an image 31-3 having reduced visualrecognition of the background other than the face and having improvedvisual recognition of the face is obtained. In the case where the inputimage is not a sharp image, the image processing section 1-3 improvesthe visual recognition of the face by processing the portion of theinput image corresponding to the white-pixel region of the face mask by,for example, edge sharpening. As a result, an image 31-4 is obtained.Similar effects may be obtained by reducing the contrast of the image,instead of reducing the sharpness of the regions other than the faceregion. In the case where the input image is a low-contrast image,similar effects may be obtained by increasing the contrast of theface-skin region. Alternatively, the contrast of the whole input imagemay be increased so that the portion of the input image corresponding tothe white-pixel region of the face mask has the highest contrast.

[0144] According to the present invention, the user roughly designates aposition (or a region) of the object in the original image, whereby animage which includes the object at a well-balanced position can be cutout from the original image.

[0145] In one example, the user roughly designates a position (or aregion) of the object in the original image, whereby an image having aprescribed size and including the object at a well-balanced position canbe output.

[0146] In one example, the user roughly designates a position (or aregion) of the person's face in the image, whereby a region representinga face skin can be extracted.

[0147] In one example, the user roughly designates a position (or aregion) of the person's face in the image, whereby a rectangle includinga region representing the face skin can be obtained.

[0148] In one example, the user roughly designates a position (or aregion) of the person's face in the image, whereby the central axis ofthe face can be detected.

[0149] In one example, the user roughly designates a position (or aregion) of the person's face in the image, whereby a vertical positionof the nose in the face can be detected.

[0150] In one example, the user roughly designates a position (or aregion) of the person's face in the image, whereby a vertical positionof the eyes in the face can be detected.

[0151] In one example, the user roughly designates a position (or aregion) of the person's face in the image, whereby a vertical positionof the mouth in the face can be detected.

[0152] In one example, the user roughly designates a position (or aregion) of the person 's face in the image, whereby a width of the facecan be detected.

[0153] In one example, the user roughly designates a position (or aregion) of the person's face in the original image, whereby an imagewhich includes the face at a well-balanced position can be cut out fromthe original image.

[0154] In one example, the user roughly designates a position (or aregion) of the person's face in the image, whereby an image having aprescribed size and including the face at a well-balanced position canbe output.

[0155] In one example, the user roughly designates a position (or aregion) of the person's face in the image, whereby the image quality canbe adjusted so that the visual recognition of the face is improved.

[0156] Various other modifications will be apparent to and can bereadily made by those skilled in the art without departing from thescope and spirit of this invention. Accordingly, it is not intended thatthe scope of the claims appended hereto be limited to the description asset forth herein, but rather that the claims be broadly construed.

What is claimed is:
 1. An image processing apparatus, comprising: adesignating section for designating an arbitrary region or an arbitraryposition of an image; a specifying section for specifying an objectregion which is present in the designated region or position, and whichcan additionally be in a vicinity of the designated region or position,from pixel information in the designated region or position and pixelinformation which can additionally be in the vicinity of the designatedregion; a determining section for determining an image region to be cutout from the image, based on the specified object region; and a cuttingsection for cutting out the determined image region from the image. 2.An image processing apparatus according to claim 1 , wherein thedetermining section includes a section for adjusting a size of the imageregion to a prescribed size.
 3. An image processing apparatus accordingto claim 1 , wherein the determining section includes a correctingsection for entirely correcting the designated image region orcorrecting only a part of the designated image region.
 4. An imageprocessing apparatus, comprising: a designating section for designatingan arbitrary region or an arbitrary position of an image; an analyzingsection for analyzing a color distribution in the designated region orposition and in a vicinity of the designated region or position; anadjusting section for adjusting a condition for specifying a face imagewhich is present in the image, according to a result of the analysis; aspecifying section for specifying a face image region which is presentin the designated region or position, and which can additionally be inthe vicinity of the designated region or position, based on the adjustedcondition; a determining section for determining an image region to becut out from the image, based on the specified face image region; and acutting section for cutting out the determined image region from theimage.
 5. An image processing apparatus according to claim 4 , whereinthe determining section includes a section for adjusting a size of theimage region, using the region or the position designated by thedesignating section as a reference.
 6. An image processing apparatusaccording to claim 4 , wherein the specifying section includes a sectionfor applying noise elimination or labelling to the specified face imageregion to produce a face mask; a section for vertically scanning theproduced face mask to obtain a sum of vertical differential luminancevalues of pixels in the image corresponding to the face mask to producea histogram; and a section for detecting a central axis of a face from aprofile of the produced histogram.
 7. An image processing apparatusaccording to claim 4 , wherein the specifying section includes a sectionfor applying noise elimination or labelling to the specified face imageregion to produce a face mask; a section for vertically scanning theproduced face mask to obtain a mean luminance value of pixels in theimage corresponding to the face mask to produce a histogram; and asection for detecting a vertical nose position from a profile of theproduced histogram.
 8. An image processing apparatus according to claim4 , wherein the specifying section includes a section for applying noiseelimination or labelling to the specified face image region to produce aface mask; a section for horizontally scanning the produced face mask toobtain a mean luminance value of pixels in the image corresponding tothe face mask to produce a histogram; and a section for detecting avertical eye position from a profile of the produced histogram.
 9. Animage processing apparatus according to claim 4 , wherein the specifyingsection includes a section for applying noise elimination or labellingto the specified face image region to produce a face mask; a section forhorizontally scanning the produced face mask to obtain a mean luminancevalue of pixels in the image corresponding to the face mask to produce ahistogram; and a section for detecting a vertical mouth position from aprofile of the produced histogram.
 10. An image processing apparatusaccording to claim 9 , wherein the specifying section further includes asection for detecting a vertical eye position from the profile of theproduced histogram; and a section for obtaining a middle position of aregion between the detected vertical eye position and the detectedvertical mouth position to detect a width of the face mask at the middleposition.
 11. An image processing apparatus according to claim 4 ,wherein the determining section includes a section for adjusting aposition of the image region, based on the face image region, a centralaxis of a face in the face image, a vertical nose position of the facein the face image, a vertical eye position of the face in the faceimage, a vertical mouth position of the face in the face image, and awidth of a face mask of the face image.
 12. An image processingapparatus according to claim 4 , wherein the determining sectionincludes a section for adjusting a size of the image region, based onthe face image region, a central axis of a face in the face image, avertical nose position of the face in the face image, a vertical eyeposition of the face in the face image, a vertical mouth position of theface in the face image, and a width of a face mask of the face image.13. An image processing apparatus according to claim 4 , wherein thedetermining section includes a correcting section for entirelycorrecting the designated image region or correcting only a part of thedesignated image region.